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Abstract — In this paper, we present a probabilistic analysis 
of iterative node-based verification-based (NB-VB) recovery 
algorithms over irregular graphs in the context of compressed 
sensing. Verification-based algorithms are particularly inter- 
esting due to their low complexity (linear in the signal dimen- 
sion n). The analysis predicts the average fraction of unverified 
signal elements at each iteration I where the average is taken 
over the ensembles of input signals and sensing matrices. The 
analysis is asymptotic (n — > oo) and is similar in nature to 
the well-known density evolution technique commonly used 
to analyze iterative decoding algorithms. Compared to the 
existing technique for the analysis of NB-VB algorithms, 
which is based on numerically solving a large system of 
coupled differential equations, the proposed method is much 
simpler and more accurate. This allows us to design irregular 
sensing graphs for such recovery algorithms. The designed 
irregular graphs outperform the corresponding regular graphs 
substantially. For example, for the same recovery complexity 
per iteration, we design irregular graphs that can recover up 
to about 40% more non-zero signal elements compared to the 
regular graphs. Simulation results are also provided which 
demonstrate that the proposed asymptotic analysis matches 
the performance of recovery algorithms for large but finite 
values of n. 

I. Introduction 

Consider a signal v G R" with only k non-zero elements. 
Let m be a positive integer so that k < m <C n. The 
main idea in compressed sensing is to represent v with 
measurements c 6 R m (measuring process), and to be able 
to recover back the original signal v from the measurements 
c (recovery process) (T), (2). 

The measuring process is essentially a linear transforma- 
tion that can be represented by the matrix multiplication 
c = Gv, or equivalently characterized by a bipartite 
graph J3J. In these representations, G is referred to as the 
sensing matrix and the bipartite graph as the sensing graph, 
respectively. The recovery process is essentially estimating 
v based on the knowledge of c and G and is considered 
successful if v is estimated correctly 

If the density of edges in the sensing graph is high, the 
computational complexity of finding the measurements and 
that of the recovery is considerably higher compared to the 
cases where the sensing graph is sparse. The high complex- 
ity of recovery for dense graphs hinders their application 
to high-dimensional signal recovery (signals with large n). 
Moreover, in certain compressed sensing applications such 
as computer networks |4|-[6|, channel coding [7|, spectrum 
sensing J8j, and identification of linear operators |9j, the 
nature of the problem results in a formulation with a sparse 



sensing graph. The main focus of this paper is on such 
problems. More specifically, our main interest is in a sub- 
class of message-passing recovery algorithms, called Node- 
Based Verification-Based (NB-VB) algorithms fT0)-p3), 
and the performance of these algorithms over irregular 
sparse graphs (matrices). The VB recovery algorithms, in 
general, are iterative and have computational complexity 
0(n), which makes them suitable for applications involving 
recovery of signals with large n. Moreover, if certain 
conditions are satisfied, the performance of VB algorithms 
is not sensitive to the distribution of non-zero elements 
in the sensing matrix and the original signal ]T2) , fl3) . 
Another interesting feature of VB algorithms is that their 
performance can be analyzed in the asymptotic case (n — > 
oo) fT0)-p"5). These properties make the VB algorithms 
a suitable choice for low-complexity recovery of sparse 
signals. For a comprehensive study on the VB algorithms, 
we refer the interested readers to (12), fl6| . 

The VB algorithms are, however, sensitive to the pres- 
ence of noise in the measured data. The authors in (T2| 
discussed the use of a thresholding technique to deal with 
noisy measurements. This technique is very effective in 
the high signal to noise ratio (SNR) regime (such as the 
scenario in Q). Having said that, some compressed sensing 
applications (see [5|, [6] for some examples) are formulated 
as noiseless problems with sparse measurement matrices. 
Furthermore, the noise-free analysis of recovery algorithms 
serves as an upper bound for the performance of the noisy 
versions. All of the above highlight the importance of noise- 
free analysis of the recovery algorithms in compressed 
sensing, in general, and in compressed sensing with sparse 
measurement matrices, in particular. 

The main focus of this paper is to analyze NB-VB 
recovery algorithms for compressed sensing problems with 
irregular sensing graphs and to design sensing graphs that 
perform well with these recovery algorithms. Our results 
are derived in the asymptotic regime (n — > oo). In this 
regime, we assume a probabilistic model for the input 
signal, in which a signal element is zero with probability 
1 — a or takes a value from a continuous distribution with 
probability a. Henceforth, the parameter a is referred to 
as the density factor. Let denote the probability that 
a signal element is non-zero and unknown at iteration £ 
over the ensemble of all sensing graphs and all inputs of 
interest. In the asymptotic regime, the recovery algorithm 
is called successful for the initial density factor = a if 



and only if Hindoo a"' = 0. Indeed, if the initial density 
factor is smaller than a certain threshold, referred to as the 
success threshold, then the recovery algorithm is successful 
as n — » oo and £ — > oo |10]-|15|. Fixing the compression 



ratio m/n, it is desirable to devise sensing graphs with the 
highest success threshold possible. 

An asymptotic analysis of NB-VB algorithms was first 
presented in flO) , where a system of coupled differential 
equations had to be solved. To cope with the high complex- 
ity of solving such a systems, the authors used numerical 
methods to solve the system for finite values of n, and 
thus obtained an approximation of the asymptotic result by 
choosing a large value of n. The numerical approximation 
in fTQ i] translates to long running times and the possibility 
of numerical errors propagating through the iterations in the 
analysis. The latter would compromise the accuracy of the 
obtained success thresholds fl2) , | fT3| . 

In (12) , (13) , the authors developed a low-complexity 
framework for the asymptotic analysis of NB-VB algo- 
rithms over sparse random regular sensing graphs. The 
analysis presented in p2| , (T3) was significantly faster 
and more robust against numerical errors compared to the 
approach of (TO). 



In this paper, we extend the analysis presented in |12|, 
(T3) to irregular graphs. Our simulations show that for a 
given compression ratio m/n, irregular graphs can provide 
up to 40% larger success thresholds compared to regular 
graphs. In this comparison, since the number of edges in 
both graphs is the same, the recovery complexity remains 
almost the same. Just like the analysis in (12) , (T3J, the pro- 
posed analysis is developed for noiseless measurements and 
its computational complexity increases only linearly with 
the number of iterations. Moreover, the analysis is simple to 
perform, requiring only additions and multiplications. This 
makes it possible to use the analysis at the core of a process 
to design degree distributions for irregular sensing graphs 
that perform well with NB-VB algorithms. The performance 
measure considered in this work is the success threshold. 

II. Ensembles of Sensing Graphs and Inputs 

Let Q(V U C, E) denote a bipartite graph or a bigraph 
with the node set V U C and the edge set E, so that every 
edge in E connects a node in V to a node in C. Further, let 
A (Q) denote the biadjacency matrix of graph Q; the entry 
ajj in A is 1 if there exists an edge connecting the nodes 
Cj £ C and Vj G V. Following the coding terminology, 
we refer to the sets V and C as variable nodes and check 
nodes, respectively. 

In general, a bigraph can be weighted and irregular. In 
the weighted bigraph Q'(V U C,W{E)) a weight w l3 := 
w(eij) £ K\{0} is associated with each edge e,j £ E. 
The weight also appears in the (i,i)th entry of the 
biadjacency matrix A(Q') corresponding to the weighted 
bigraph Q' . 

In a bigraph, a node in V (C) has degree i if it is 
neighbor (connected) to i nodes in C (V). Let Xi £ M. + 
and pi £ R + denote the fraction of nodes in V and C with 



degree i, respectively. The polynomials X(x) — J^i ^i xl 
and p(x) — Y^i Pi x% are referred to as degree distributions 
corresponding to nodes in V and C, respectively. Clearly, 
A(l) = p(\) = 1. For mathematical convenience, we define 
d v := i\ an d d c := . 3Pj an d we refer to them as 
the average variable degree and the average check degree, 
respectively. 

For given X(x),p(x) and n, let Q n (X(x), p{x)) (G n (X, p) 
for short) denote the ensemble of all irregular bigraphs with 
n variable nodes and degree distributions X{x) and p(x). 
Further, let yy™ x " be the ensemble of all m x n (to — 
nd v /d c ) matrices with i.i.d. entries w drawn according to 
a distribution f(w). Now, for any irregular bigraph Q(V U 
C, E) £ g n (X, p) and any weight matrix W £ W™ xn , we 
form the corresponding (n, A, p)-weighted irregular bigraph 
G'(VU C, W{E)) as follows. Let us assume an arbitrary, 
but fixed, labeling scheme for node sets V and C over the 
ensemble Q. To every edge e»j £ E, 1 < i < to, 1 < 
j < n, connecting Cj £ C and Vj £ V, we assign the 
weight in row i and column j of the weight matrix W; 
i.e., w(eij) — Wij. Thus, we construct the ensemble of all 
(n, X, p)- weighted irregular bigraphs, denoted by Q r j(X,p), 
by freely combining elements in Q n (X,p) and W'™ x ™. 

To describe the inputs of interest, let a £ [0, 1] be a fixed 
real number and v be a vector of length n with elements 
Vi drawn i.i.d. according to Pr[vi = v] = ctg(v) + (1 — 
a)5(v), where S(-) and g(-) are the Kronecker delta and 
a probability density function, respectively. We denote the 
ensemble of all such vectors by V™(a)^J 

To build the compressed sensing setup, let G(V U 
C, W(E)) be a weighted irregular bigraph drawn uniformly 
at random from the ensemble Q 7 j(X,p) with G as its 
biadjacency matrix. Moreover, let v be a signal vector 
drawn uniformly at random from the ensemble V™(a). 
Also, let c = Gv. The sets of signal elements v and 
measurements c are respectively mapped to the vertex sets 
V and C (|V| = n, \C\ = m) with the sensing matrix being 
the biadjacency matrix G. Henceforth, we refer to the graph 
Q as the sensing graph. 

III. VB Algorithms and Verification Rules 

As a class of iterative recovery algorithms, the VB 
algorithms find the value of a set of signal elements in each 
iteration based on the knowledge of the measurements, the 
sensing graph and the previously verified signal elements. In 
||TTJ— (T3J, (17), (18), it was demonstrated that the continuity 
of at least one of the distributions / or g (non-zero weights 
of the sensing graph or non-zero signal elements), is a 
sufficient condition for the assigned value to a variable node 
at a certain iteration to be its true value with probability one 
(zero probability of false verification). In this paper also, we 
assume that the probability of false verification throughout 
the recovery algorithm is zero. 

It is worth noting that the expected fraction of non-zero elements in 
such a vector is a. Using a Chernoff bound, it can be shown that the 
actual fraction of non-zero elements in a randomly chosen vector from 
this ensemble is tightly concentrated around its expected value (a) with 
high probability. 



The algorithm discussed in [IT], which is the same as 
the LM2 algorithm of (TTJ, performs the best among all 
known VB algorithms in the context of compressed sensing 
(10), (12), (13). We refer to this algorithm as SBB. The 
analysis of SBB is the focus in this work. The proposed 
analytical framework is, however, general and applicable to 
the analysis of other NB-VB algorithms. In what follows, 
we discuss the verification rules of the SBB algorithm. 

When a variable node is verified at an iteration of an NB- 
VB algorithm, its value is subtracted from the value of all 
its neighboring check nodes. Then, the variable node and 
all its adjacent edges are removed from the sensing bigraph. 
Consequently, all the check nodes neighbor to the verified 
variable node face a reduction in their degree by one. The 
algorithm stops at an iteration if the set of verified variable 
nodes remains unchanged for two consecutive iterations or 
when all the variables are verified. A variable node in SBB 
is verified based on the following verification rules. 

• Zero Check Node (ZCN): All variable nodes neighbor 
to a zero-valued check node are verified with a value 
equal to zero. 

• Degree One Check Node (D1CN): If a variable node 
is neighbor to a check node of degree 1, it is verified 
with the value of the check node. 

• Equal Check Nodes (ECN): Let C denote a set of check 
nodes with the same non-zero value. Also, let V denote 
the set of all variable nodes neighbor to at least one 
check node in C. Then, 1) a variable node in V which 
is not connected to all check nodes in C is verified 
with a value equal to zero; 2) if there exists a unique 
variable node in V that is neighbor to all check nodes 
in C, then it is verified with the common value of the 
check nodes. 

In order to introduce the asymptotic analysis of NB-VB 
algorithms and prove the concentration results, the authors 
in (T2) , (L3) developed a message passing representation of 
VB algorithms. In this paper, we use the same representa- 
tion and refer the interested reader to the aforementioned 
references for more details. 

IV. Asymptotic Analysis Framework 

Let the probability distributions / and g, the degree 
distributions A and p, and the density factor a be fixed. 
It can be shown that the fraction of unverified non-zero 
variable nodes at each iteration £ of the SBB algorithm 
(a") over a realization of the sensing graph Q £ Qf(^, p) 
and a realization of the input signal V £ V™ (a) concentrates 
around the average of a^' taken over all the elements in 
the ensemble 5y(A,p) x Vg(a), as n tends to infinity^] 
The deterministic analysis presented here is to track the 
evolution of this average as n goes to infinity. In the 
language of coding, the analysis is similar to the density 
evolution analysis of iterative decoding algorithms over 
irregular LDPC code ensembles, with the main difference 

2 These concentration results have been proved in detail for regular 
graphs in |12| . Similar results apply to the irregular sensing graphs with 
minor changes, and are therefore not presented in this paper. 



TABLE I 

Sets that change in each half-round of each round at any 
iteration 



Rl 


R2 


HR1 


HR2 


HR1 


HR2 


Mm A4j 


K-i -> Kj 


K,k -> Mj,k 


A,; -> Aj 



being that the NB-VB algorithms do not conform to the 
principle of extrinsic message passing which significantly 
simplifies the density evolution analysis in the context of 
coding. 

The mathematical framework for the analysis is similar to 
the one used in p2"l , (T3J, however with two extra variables 
d v and d c which represent the degree of a variable and 
a check node, respectively. These variables take different 
values for irregular graphs while for a regular graph they 
each have a fixed value. This makes the derivations more 
tedious. Due to the lack of space, in the following, we only 
provide a sketch of the analysis. 

At the beginning of each iteration £, the analysis par- 
titions the set of all unverified variable nodes into two 
(disjoint) sets: non-zero variable nodes (/C^), zero-valued 
variable nodes (AW). Should the fraction of variable nodes 
in the set JO?) tend to zero as iterations proceed, the fraction 
of variable nodes in the set A'^ will also tend to zero and 
consequently the analysis declares a successful recovery 

G3- 

Each iteration in the SBB algorithm is divided into two 
rounds (R), each consisting of two half-rounds (HR). In 
the first and second rounds, verified variable nodes belong 
to the sets K.^ and A«, respectively. The configuration 
of the sets at the end of each half-round is specified using 
the superscript (£,Rx,y), where £, x <E {1,2} and y 6 
{1,2} denote the iteration, round and half-round numbers, 
respectively. 

We partition the set of all check nodes with the same 
degree (say d c ) into sets Af-j(d c ),0 < i < d c ,0 < j < 
d c — i, where i and j indicate the number of neighboring 
variable nodes in the sets JO?' and A'*', respectively. 

Let JO?'{d v ) and A^(d v ) denote the set of all non- 
zero and zero-valued unverified variable nodes with the 
same degree d v , respectively. Then, the set JO^{d v ) is 
further divided into subsets (d v ),0 < i < d v , where 
i denotes the number of neighboring check nodes in the 
set Aff ] := U dc Uio 1 -^'* 1 ' 1 ^)- Als °> we divi de the 
set A^(d v ) into subsets A^(d v ),0 < i < d v , with 
the following definition: a variable node in A^ has i 
neighboring check nodes which became zero-valued after 
HR1 of R2. Table UJ summarizes the sets affected in each 
half-round of each round at any iteration. 

Theorems [TJ and [2] below, characterize the verification of 
unverified non-zero (JC^) and zero-valued (Aw) variable 
nodes at HR2-R1 and HR2-R2 in each iteration £ of the 
SBB algorithm, respectively. The proofs of the theorems 



are very similar to the ones presented in [12| and therefore 
omitted. 

Theorem 1. In the first round of any iteration i, a non- 
zero variable node of degree d v is verified if and only if it 



TABLE II 

Success Thresholds for Regular, Left-Regular, _ 
Right-Regular, and Bi-Irregular Graphs with <L, = 4, d r 



belongs to the set |jf=2 1cf' R1 ' 2 ^ U IC\"" JX1 ' Z> , where the set 
£(e,m,2) cons j sts D f a n variable nodes in the set JC^' R1 '' 2 ^ 
connected to the set A/^ Q fll . 

Theorem 2. In the second round of any iteration £, a zero- 
valued variable node of degree d v is verified if and only if 
it belongs to the set U^=i ^f 1 ■ 

The sets M e \ J\fi t 7 1,m,x 



K^- w \ and 



1 > m ' 2 > f u iiy describe the state of the algorithm at the 
beginning of iteration t. The probability that a variable node 
belongs to the set K."' is aS l \ 

The asymptotic analysis tracks the probability that a 
node (variable node or check node) belongs to a certain 
set at each half-round, round, or iteration. The recovery is 
successful if and only if the probability tends to zero, 
as £ tends to infinity. The analysis is based on the derivation 
of recursive equations that relate the probabilities described 
above for two consecutive iterations. The complexity of the 
analysis thus scales linearly with the number of iterations. 

V. Simulation Results 

To verify the asymptotic results obtained based on the 
analysis of Section IV we perform some finite-length 



simulations for large values of n. The input signal in all 
simulations follows the probabilistic model described in 
Section [II] Also, each non-zero signal element is drawn 
from a standard Gaussian distribution (zero-mean with 
variance one). The graphs are constructed randomly with 
no parallel edges and all edge weights are chosen to be 
1. Each simulation point is generated by averaging over 
1000 random instances of the input signal. In simulations, 
the recovery is successful if and only if the input signal 
is recovered perfectly. For the analytical results, based on 
the fact that a"' is a non-increasing function of iteration 
number £, we consider the following stopping criteria: 

1) Success: < 10~ 7 . 

2) Failure: a® > 10~ 7 and |«W - a^ -x )| < 10~ 8 . 
To calculate the success threshold, a binary search is 

performed within a certain range of initial density factors 
which includes the threshold. The search continues until the 
search region is smaller than 10~ 5 . 

To motivate the use of irregular graphs for the purpose 
of sparse signal recovery, we first present some comparison 
with regular graphs. In Table |llj we compare the success 
threshold of the SBB algorithm over regular, left-regular 
(all variable nodes have the same degree), right-regular 
(all check nodes have the same degree) and bi-irregular 
graphs, when the average variable and check degrees are 
fixed at d v = A,d c = 5, respectively. The success 
thresholds reported for the left- and right-regular graphs 
are the highest thresholds obtained by optimizing right 
and left degree distributions with maximum degree 20 and 



Graph Type 


Success Threshold 


Improvement (%) 


Regular 


0.4225 




Right-Regular 


0.5319 


25.89 


Left-Regular 


0.5247 


24.16 


Bi-Irregular 


0.5795 


37.15 



with up to four non-zero components, respectively. For 
the bi-irregular case, however, we restricted the search to 
only two degrees (bimodal distribution) both less than 20 
for both variable and check nodes. The optimized degree 
distributions (X(x),p(x)) are: (0.9310a; 3 + 0.0350a; 17 + 
0.0340x 18 , a; 5 ), (a; 4 , 0.7100a; 3 +0.1830a; 5 +0.1070x 20 ), and 
(0.9000a; 3 + 0.1000a; 13 , 0.9375a; 4 + 0.0625a; 20 ) for right- 
regular, left-regular, and bi-irregular graphs, respectively. 
As expected, the bi-irregular graphs achieve the highest 
success threshold, almost 37% higher than that of the 
regular graphs. 

To investigate the degree of agreement between our 
asymptotic analysis and finite-length simulations, we have 
presented in Fig. [T] the evolution of (for the theoret- 
ical results) and the average unverified non-zero variable 
nodes normalized by n (for the finite-length results) with 
iterations I for the SBB algorithm. The sensing graph is a 
randomly constructed bi-irregular graph with the optimized 
degree distributions X(x) = 0.9000a; 3 + 0.1000a; 13 and 
p(x) = 0.9375x 4 +0.0625a; 20 with 10 5 variable nodes. Two 
values of cr°) are selected: one above the success threshold 
(0.595 > 0.5795) and one below it (0.575 < 0.5795). 
The theoretical results are shown by dotted lines while 
simulations for n = 10 5 are presented with solid lines. As 
one can see, the two sets of results are in close agreement 
particularly for the cases where cr -* is above the threshold 
and also for smaller values of I. 
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Fig. 1. Evolution of a", obtained by the theoretical analysis, vs. iteration 
number i (dotted line) and that of the normalized average unverified non- 
zero variable nodes vs. t for n = 10 5 (solid line). 

In Figure [2] we have plotted the average normalized 
number of unverified variables (success ratio) of SBB over 



TABLE III 

Optimal Left Degree Distributions and the Corresponding 
Thresholds a* of SBB for Right-Regular Graphs with 
d v = 4 and Different Check Degrees Along with the 



d c 


Optimum A(a;) 


a* 


a *R 


5 


0.931a; 3 + 0.035a; 17 + 0.034a; 18 


0.5319 


0.4225 


6 


0.917a; 3 + 0.082a; 15 + 0.001a; 19 


0.4137 


0.3387 


7 


0.906a; 3 + 0.034a; 13 + 0.06a; 14 


0.3369 


0.2811 


8 


0.896a; 3 + 0.04a; 12 + 0.064a; 13 


0.2831 


0.2394 



TABLE IV 
of sbb ove 

with Optimal Bimodal Left Degree Distributions Along 

with the Bimodal Degree Distributions (a* is the SBB 
Threshold of a Right-Regular Graph with Optimal Left 
Degree Distribution with Up to 4 non-zero Components) 



(d v ,d c ) 


X(x) Bi-Modal 


a* B 


a* 


(4,5) 


0.9333a; 3 + 0.0667a- 18 


0.5318 


0.5319 


(4,6) 


0.9166a; 3 + 0.0834a; 15 


0.4137 


0.4137 


(5,9) 


0.8750a; 3 + 0.1250a; 19 


0.3429 


0.3431 


(5,10) 


0.8571a; 3 + 0.1429a; 17 


0.3001 


0.3017 



the graphs with optimized right-regular and bi-irregular 
degree distributions of Table [IT] vs. the initial density factor. 
Each graph has n = 10 5 variable nodes. We note that 
the success threshold for each graph, demonstrated by a 
vertical line on the figure, matches the waterfall region of 
the corresponding finite-length simulations. 
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Fig. 2. Verifying the success threshold of SBB over right-regular and 
bi-irregular graphs of Table |ll] through finite-length simulations. 

More optimization results for right-regular graphs with 
d v — 4 and different d c values are presented in Table 



III For comparison, the threshold values a* R of the cor- 
responding regular graphs are also given in the table. An 
inspection of the threshold values reveals an improvement 
of about 18% to 26% in the threshold of irregular graphs 
in comparison with that of regular graphs. 

To reduce the design complexity of irregular graphs, we 
also investigate the effect of reducing the number of non- 
zero components in the left degree distribution of right- 
regular graphs. The results for some optimal bimodal left 
degrees are presented in Table [IV] It can be seen from 
the table that the difference between the thresholds a B of 
bimodal left degrees and the thresholds a* of left degrees 
with up to 4 nonzero components are negligible, i.e., only 
two different degrees on the variable side of a right-regular 
sensing graph suffices to provide very good performance. 
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